NetNews Offline 2/NetNews Offline Volume 2.iso/news/comp/lang/c-part1/7184

Text File | 1996-08-05 | 7.2 KB | 171 lines

Path: rcp6.elan.af.mil!rscernix!danpop
From: [e-mail] (Dan Pop)
Newsgroups: comp.os.msdos.programmer,comp.lang.c
Subject: Re: open vs fopen?
Date: 17 Feb 96 02:59:24 GMT
Organization: CERN European Lab for Particle Physics
Message-ID: <danpop.824525964@rscernix>
References: <[e-mail]> <4f8bev$[e-mail]> <[e-mail]> <4ftusv$[e-mail]> <danpop.824430285@rscernix> <4g39d4$[e-mail]>
NNTP-Posting-Host: ues5.cern.ch
X-Newsreader: NN version 6.5.0 #7 (NOV)

In <4g39d4$[e-mail]> [e-mail] (John Noland) writes:

>In article <danpop.824430285@rscernix>, [e-mail] says...
>>
>>In <4ftusv$[e-mail]> [e-mail] (John Noland) writes:
>>
>>>The open() function and its ilk are normally referred to as the "low-level"
>>>I/O package. fopen() is the "Buffered" or "Standard" I/O package. The
>>>strength of the low-level I/O functions is that they offer excellent control,
>>>particularly when used with binary files.
>>
>>???  What can read/write do on a binary file that fread/fwrite cannot do?
>
>Where in the above did I compare them with each other?

In the above paragraph.  The "low-level I/O functions" means essentially
read/write.
>>
>>>If you have a special I/O need, you
>>>can use the low-level I/O routines to fashion the exact I/O package to fit
>>>your needs.
>>
>>Same question as above.
>
>Same question from me again. Are the read/write functions written using fread/
>fwrite or is it vice versa? I believe, and feel free to correct me if I'm
>wrong, that the fread/fwrite functions are written using the read/write
>functions.

True (in Unix, probably false on other platforms) but irrelevant.  There
are no I/O needs that can be satisfied by one set of functions and not
by another, unless we start talking about Unix-specific features
(pipes, sockets, select()).
>>
>>>The standard I/O package is one such creation. It is designed to provide fast
>>>buffered I/O, mostly for text situations.
>>               ^^^^^^^^^^^^^^^^^^^^^^^^^^
>>???
>
>I use the stdio routines almost exclusively myself. Even for binary files.
>That doesn't make the above statement wrong. Your statements seem to imply
>that all systems are buffered at the OS level, so who cares what I/O
>approach you use. You're not going to see any difference in I/O speed no
>matter what. While this may be true for your particular situation, it's by
>no means universal. The usefulness of buffering is in working sequentially
>with a file. This makes buffered I/O well suited for text files. That
>doesn't mean you can't or won't read a binary file sequentially or that it
>sucks for use with binary files. Which, judging from what you've written,
>is exactly what you'll think I meant.

You meant that stdio routines are somehow more suitable for text I/O,
which doesn't happen to be true.  The stdio buffering is a red herring,
because it can be always disabled, if it isn't needed.

>>>For a lot of applications, using the
>>>standard I/O package is simpler and more effective than using simple low-level
>>>output. The essential feature is its use of automatic buffering. Buffered
>>>I/O means reading and writing data in large chunks from a file to an array
>>>and back. Reading and writing data in large chunks greatly speeds up the I/O
>>>operations, while storing in an array allows access to the individual bytes.
>>>The advantages of this approach should be apparent.
>>
>>Except that you can read and write data in large chunks using stdio
>>routines.  The impact of the extra level of buffering is insignificant.
>
>Isn't the above blurb talking about the stdio routines? Once again, your
>reference to an extra level of buffering is not applicable to all systems.
???  Do you mean there are systems where stdio doesn't perform its own
buffering?

>If you're really referring to the low-level routines, then your statement
>is not always true.

I was referring to the stdio routines.  In the (rare) cases where the
stdio buffering has a non-negligible impact, it can be simply disabled.

>>>When fopen() is used, several things happen. The file, of course, is opened.
>>>Second, an external character array is created to act as a buffer. The
>>><stdio.h> file has this buffer set to a size of 512 bytes.
>>
>>To a default size of _minimum_ 256 bytes.  I'm typing this text on a
>>system which uses a default buffer size of 8192 bytes.  The size of the
>>buffer can be controlled by the programmer via the setvbuf function.
>>
>That's a big default buffer. How many disk accesses does it take to fill
>that thing?

If its size matches the size of the lower level buffer, the number of
disk accesses is irrelevant, because the same number will be performed
even if I ask read() to read 1 byte from the file.  The implementor is
the one who actually _knows_ what is the optimal size.  And the implementor
decided that the st_blksize field of the stat struct should be set to
(pure coincidence :-) 8192.  For the Unix illiterate:

    ues5:~/tmp 201> grep st_blksize /usr/include/sys/stat.h
    ulong_t st_blksize;  /* Optimal blocksize for file system i/o ops */

>I seriously doubt you would want a buffer that big when reading
>a file randomly. But, maybe your system is so fast you can afford that kind
>of waste. In my <stdio.h> file, the constant BUFSIZ is 512.

Aha, I see where the magical value 512 came from.  The systems I'm working
on have BUFSIZ set to values between 1024 (mostly BSD-based Unices) and
32768 (OpenVMS).  So, your claim that <stdio.h> sets the size of the
buffer to 512 bytes is bogus.  BTW, even 512 is a humongous size for a
single density 8" floppy disk, which needs to read four sectors to fill
such a buffer.

>You say that
>you can control the size of the buffer using setvbuf(). That's true, but it
>doesn't change the default buffer size as you seem to imply.

Huh???  Where did I imply that setvbuf changes the _default_ buffer size.

>If I do the following:
>
>FILE *input, *output;
>
>input = fopen("myfile.in", "r+b");
>setvbuf(input, NULL, _IOFBF, 1024);
>
>output = fopen("myfile.out", "w+b");
>
>do you think output points to a buffer of the size set by setvbuf?

Which should I think such a stupidity?

>I don't think this is what you meant.

And I _definitely_ didn't mean that.

>>>Technically, in DOS, all I/O is buffered since DOS itself uses buffers
>>>for disk I/O.
>>
>>The same applies to other operating systems (e.g. Unix) except that they
>>do a much better job at buffering I/O than MSDOS.
>
>Some versions of UNIX don't do any buffering at all.

Care to mention a few examples?  Unix has had block devices for as long
as I can remember.  And I have yet to see a disk which can perform a
1-byte write.  But wait, you can always read a full sector in memory,
change the byte and write back the sector immediately.  Voila, Unix without
buffering.  Performance?  Who cares about performance???

>>>Hope this clarifies things.
>>
>>So many inaccurate statements don't clarify things.  On the contrary.
>
>The shit you have to up with when trying to be helpful.

And when you don't succeed.  Spreading misinformation doesn't help anybody.

Dan
--
Dan Pop
CERN, CN Division
Email: [e-mail]
Mail:  CERN - PPE, Bat. 31 R-004, CH-1211 Geneve 23, Switzerland